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ABSTRACT 

A year after graduation, a survey was attempted by 
the Massachusetts Board of Regents of all 1981-82 career graduates of 
public community colleges in the state (N=5 # 267). Of the 1,881 
responding graduates, 847 reported that they were employed full-time, 
and 737 of the working graduates gave the board permission to contact 
their employers for an assessment of their performance. The employers 
of the 737 graduates were surveyed using the Employer Follow-Up (EFU) 
questionnaire developed as part of the Vocational Education Data 
System (VEDS). Study findings, based on a 79.8% response rate, 
included the following: (1) overall, the graduates were rated between 
good and very good by their employers with respect to technical 
knowledge, work attitudes, work quality, technical skills, overall 
quality, and relative preparation, as compared to co-workers who 
graduated from similar programs; (2) graduates were rated lowest on 
technical skills and technical knowledge; (3) graduates who were 
working full time in their area of training were rated significantly 
higher on technical knowledge and overall quality than graduates 
working in areas not related to their training; (4) Technology and 
Human Services graduates working in a field related to their training 
received higher rating3 than other graduates, with Medical/Health 
Science graduates having the lowest average ratings; and (5) overall, 
males and females received similar performance ratings, thovgh males 
had significantly higher average wages than females. Comments on the 
labor- and time-intensive character of the EFU are included. (ALB) 
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RESULTS OF THE VEDS EMPLOYER FOLLOW-UP SURVEY FOR 
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James Carifio, University of Lowell 

Ronald Biron, Massachusetts Board of Regents 

Allen Shwedel, Boston Public Schools 



Abstract 

This paper presents the results of the mandated VEDS 
employer follow-up survey <EFU> for 15 public community 
colleges* Occupational/career program graduates who were 
employed full-time and who consented to have their employers 
contacted <N=588> were survey between one and two years after 
graduation. As demonstrated by several empirical tests, 
neither volunteerism nor response rates were a problem in 
this study. 

In general, community college graduates were rated 
between good and very good on the VEDS-requ i red quality of 
work and preparation EFU items. Significant differences 
between community colleges, program. , and the same program 
at different community colleges were found. 

No correlations were found between EFU ratings and 
average hourly wage, even within program areas, indicating 
that the average hourly wage of graduates is a flawed index 
of program or graduate quality. 

No significant differences were found between female and 
male EFU ratings, but large significant differences in 
average hourly wage were found, even within program areas. 
This finding tends to support current contentions that women 
are not getting equal pay for equal quality or work in the 
marke tp I ace . 



Paper presented at the 1988 annual conference of~the Eastern 
Educational Research Association, February, Miaimi, Florida. 



Problem Statement 

The Vocational Education Data System <VEDS> is the major source 
Of information for monitoring the impact of federal funds allocated 
for vocational education under PL94-482. and for assessing vocational 
educational trends nationally, locally, and at the state level. In 
addition to enrollment, personnel, financial, graduate, and student 
foil ow-up <SFU> data, the VEDS ' system requ i res that educat i jna I 
institutions survey those employers who have hired their graduates 
concerning the employer's satisfaction with the graduate's 
preparation, work habits, attitudes, and on-the-job performance- A 
federally specified survey and survey design, moreover, must be used 
in conducting the Employer Follow-Up <EFU> component of the VEDS 
system. 

Very few educational institutions have actually implemented the 
EFU component of the VEDS system, and very little is known about the 
actual implementation of this key component of the VEDS system. This 
paper, therefore, reports the findings of a uniform, centralized, and 
state-wide implementation of the EFU survey by the Massachusetts 
Board of Regents for Massachusetts' community colleges. 

Methodology 

There are 15 publ ic community col leges in Massachusetts which 
offer over 40 different career (occupational educational) programs. 
The VEDS EFU survey is sent to those employers of students working 
full-time one year after graduation who consent to have the survey 
sent to their employer. These particular students and employers can 
only be identified from the VEDS student follow-up <SFU> survey which 



is sent to ail degree program graduates one year after graduation. 

A VEDS SFU survey was mailed to all 1981-1982 public community 
college career program graduates <N = 5,267> by Massachusetts Board 
of Regents. Of these graduates, 1,881 (35.7/0 returned surveys and 
3,386 <64.3%> did not. 

Surveys were returned d i recti y to the Board by graduates and 
this same methodology was used in conducting the employer follow— up 
survey . 

Of the 847 graduates who reported that they wire employed full- 
time, 737 <87%> consented tc have their employers contacted (sea 
Table 1>. Of these students, employers returned 588 (79.8X) of the 
surveys sent, with 492 of these surveys being for graduates employed 
full-time in their area of training and 80 being for those who were 
employed full-time not in their area of training (see Table 2>. 
These graduates had been working in their current job an average of 
1.7 years and 88% of them were rated by their immediate supervisor. 
In all, 192 employers and 208 supervisors who did the actual ratings 
were in the sample*. 

As can be seen from Table 1, no statistically significant 
differences in the proportions of males and females were found among 
those graduates who were employed full-time, those who consented to 
have their employer contacted, and those graduates for whom EFU 
surveys were returned, although there were two tendencies towards 
significant differences <p<.10>. The same results were found in 
terms of volunteerism and return rates by program area (Table 2). 
The effects of volunteerism and response rates in the present study 
were negligible at worst, in terms of mean levels, correlations and 
generalizations. This view is supported by a study of a ten percent 



Table I: SUMMARY OF RESPONSE RATES AND RESPONSE 

PERCENTAGES BY SEX AND EMPLOYMENT STATUS 





Total 


Fema 1 es 


Ma 1 es 


Related 


1 .Popu ! at i on 


5267 


3550 


1717 








<67.4%> 


<32.6%> 




2. Returned 


1447 


1006 


471 




Fo 1 1 ow-up 


<35.4%> 


<68.1%> 


<31.9%> 




3.Emp 1 oyed 


. 847 


569 


278 


608 


f u 1 l-t i me 


<58.5%> 


<67.1JO 


<32.8%> 


<71.7%> 


4. Consented to 


737 


518 


219 


523 


EFU survey 


<87.0%> 


<2S.7%> 


<70.3%> 


<70.9%> 


5.EFU survey 


588 


436 


152 


492 


returned 


<80%> 


<74.2%> 


<25.8%> 


<83.6%> 



Comparisons of 


response rates by 


sex 


Comgar isor 


Chi-Sg. df 


D . 


1. Employed full-time 


1.62 1 


>.05 


to Consented 






2. Consented to EFU 


2.24 1 


>.05 


survey returned 






3. EFU eligible to 


7.70 1 


<.10 


EFU return 






4.SFU return to 


7.00 1 


<.10 


EFU return 






5. Population to SFU 


10.00 1 


<.10 


return 
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Table 2: SUMMARY OF RESPONSE RATES AND RESPONSE 
PERCENTAGES BY GENERAL PROGRAM AREA 



Prog ram Area 


P02- 


SFU 


Erngl . 


Con . 


Re tu rned 


Serv i ces 


179 


4t 


32 


30 


26 


Medical /Heal th 


1377 


374 


248 


214 


206 


Child Care/Food 


153 


36 


26 


24 


22 


Sec ./Bus i ness 


2073 


662 


356 


305 


219 


Techno 1 ogy 


1282 


326 


170 


152 


106 


Product i on/other 


169 


38 


15 


12 


9 


Total 


5233 


1447 


847 


737 


588 



Samp I e Compos i t i on < percentages > Compar i sons 



Comgar json 




Ch i -Sg 


df 


2i 


1 .Emp 1 oyed full 


-t i me 


0.24 


5 


>.05 


to consented 










2. Consented to 


returned 


6.60 


5 


>.05 


EFU survey 










3. Emp 1 oyed full 


-time to 


7.20 


5 


>.05 


returned EFU 


survey 








4. Population to 


returned 


13.30 


5 


>.02* 



SFU survey 



Table 3: CORRELATIONS BETWEEN EFU SURVEY ITEMS <N=588> 







TK WA WQ 


TS 


OR 


RP 


1 

▲ ■ 


TK 


1.0 .48 .65 


.74 


.73 


.40 


2. 


MA 


1.0 .71 


.46 


.60 


.36 


3. 


WQ 


1,0 


.61 


.74 


.41 


4. 


TS 




1.0 


.67 


.37 


5. 


OR 






1 ,0 


.50 


6. 


RP 








1.0 



TK=Techn ical Know! edge 
WA=Work Attitute 
WQ=Work Quality 
TS=Technical Sk i II 3 
0R=0verall Rating 

RP-Relative Quality of Preparation 

as compared to other similar employees 



random sample of non-responders to the student follow-up survey that 
was done <n=380>, which had a response rate of only 35.4%. No 
significant differences between responders and non-responders were 
found on 8 VEDS demographic and 4 VEDS dependent variables. . A model 
was developed to explain the findings of this non-responder svudy. 
Essentially, this .odel states that as sample size and the number of 
factors be.ig studied increases, non-response tends to become a 
random rather than a systematic variable due to all of the 
interactions among the many factors involved and reciprocal canceling 
out effects (see Carifio et al., 1987 for further details). 

The VEDS EFU survey consists of six questions which ask the 
employer to rate the graduate on technical knowledge, work attitude, 
work quality, technical skills, overall quality of tne graduate, and 
the relative preparation of the graduate as compared to other recent 
similar graduates employed by the company on a 1 to 5 rating scale 
where 1 was very poor and 5 was very good. There is a'so a category 
on the rating scale that allows the rater to indicate that she or he 
has no basis to make the Judgements requested. The exact wording of 
each question and the response categories for each question were 
specified by NCES and must be used as specified. 

Table 3 presents the inter-item correlations of the 6 VEDS 
EFU survey items. As can be seen from Table 3, the inter- item 
correlations ranged from .36 to .74 with the correlation between 
quality of preparation as compared to other graduates employed by the 
company and other items being the lowest. The Cronbach's alphas for 
the first 4, 5, and all 6 items were r=+.86, r=+.90, and r=+.84 
respectively. Internal consistency coefficients did not differ 
significantly by program area, and the only systematic relationship 

ERIC 8 



found between ratings and a variety of other background variables 
was a. slight positive correlation between length of time employed at 
the company and level of rating < r=+.08,p< . 10) . All of these 
f i nd i ngs i nd i cate that the raters were cons i stent but also were 
relatively discriminating in their judgements. 

Results 

Overall, all graduates on the average were rated between good and 
very good by their employers in the areas of technical knowledge 
<4.2>, work attitudes <4.4>, work quality <4.4>, technical skills 
<4.2>, overall quality <4.3> and relative preparation as compared to 
other similar recent graduates employed by the company <4.4>. 
Graduates were rated significantly lower on Technical Knowledge 
<F=3.8, df=l,587, p.<.05> and on Technical Skills <F=3.9, df=l,587, 
p<.05> than they were on the other EFU ratings. Graduates who were 
working full-time in their area of training (see Table 4) were rated 
significantly higher than graduates who were working full-time in 
areas not related to their training on Technical Knowledge <p<.05> 
and Overall Quality <p<„03> with a tendency towards significant 
differences on Technical Skills <p<.10> and Work Quality <p<.10>. 
These differences are in the direction expected and support the 
va I i d i ty of the resu I ts obta i ned in th i s survey as those graduates 
who are working full-time in areas not related to their training are 
an internal control group and this is the reason why they are 
included in the EFU survey by NCES. 

Significant differences between community colleges, programs 
<see Table 5> and between the same program at different colleges 
q across the state were found from the .05 to the .001 level. In 



Table 4: MEAN EFU RATINGS OF GRAOUATES WORKING IN THEIR AREA 
- OF TRAINING AND GRADUATES WORKING IN AREAS UNRELATED 
TO THEIR TRAINING <N=588>. 



N TK WA WQ IS OR RP 
Related 492 4.2 4.4 4.4 4.2 4.3 4.4 

Not Related 80 4.0 4.3 4.2 4.1 4.0 4.3 



F 3.8* 1.2 2.9 2.8 5.1* 0.1 

p <.05 >.05 <.10 <.10 <.05 >.05 



Table 5: MEAN EFU RATINGS OF GRADUATES WORKING IN THEIR AREA 
OF TRAINING BY GENERAL PROGRAM AREA (N=492> 



Program Area N TK WA WO, 

Services 17 4.0 4?4 4.4 

Medical/Heal th 173 4.1 4.2* 4.3 

Child Care/Food 19 4.3* 4.4 4.7 

Sec. /Business 189 4.2 4.5 4.5 

Technology 88 4.4* 4.5 4.5 

Production 6 4.1 4.6 4.3 



IS 
4.0 
4.0 

4.3* 4.5 
4.1 4.4 
4.4* 4.4 
4.1 4.3 



OR RP 



4.1* 4.4 
4.1* 3.7* 
4.7 
4.6 
4.7 
4.6 



Total 



F 
P 



492 4.2 4.4 4.4 4.2 4.3 4.4 

2.6* 3.5* 2.2 2.5* 3.2*13.5* 
<.05 <.01 >.05 >.05 <.05 <.001 



Table 6: MEAN EFU RATINGS OF FEMALE AND MALE GRADUATES 
WORKING IN THEIR AREA OF TRAINING <N=492> 



Fema I es 
Ma I es 



N TK 
384 4.2 
148 4.3 



WA 
4.4 
4.5 



wg 

4.4 
4.4 



TS 
4.2 
4.1 



OR RP 
4.2 4..5 
4.2 4.2 



F 
P 



0 

ERIC 



1.1 0.9 0.1 0.7 0.2 4.6* 
>.05 >.05 >.05 >.05 >.05 <.05 
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general, graduates working full-time in their area of training in 
Technology and Human Services programs were rated significantly 
higher on the EFU survey items than other program areas. Graduates 
in Med i ca i /Hea I fch Science programs were overall rated significantly 
lower than other students and had the lowest average rating on 
Relative Preparation (somewhat good) of al! program areas (see Table 
5). These findings, it should be noted, ran parallel to th«a margina 
but not significant differences in employer contact consent rates. 

The findings above wore a significant factor in the Board of 
Regents' decision to conduct a state-wide review of two year 
Med ica I/Heal th programs as well as selected programs in other areas 
at particular community colleges. The overall significance and 
importance of those findings, however, can only be understood in 
relationship to findings concerning the relationships between sex, 
average hourly wage, program aroas, and EFU ratings. 

Table 6 presents the mean EFU ratings of female and male 
graduates working in their area of training. As can be seen from 
Table 6, no significant differences were found between female and 
male EFU ratings with the exception of relative preparation as 
compared to other similar recent graduates, where males were rated 
significantly higher than females <p<.05>. This difference is due 
to the low ratings of the Med i ca I /Hea I th programs which are almost 
exclusive female. When this program area was eliminated from the 
analyses, no difference between female and male ratings were found. 

Table 7 presents the average hourly wages for females and males 
As can be seen from Table 7, males had significantly higher average 
hourly wages than females <p<«001> f even though no significant 
differences were found between females and males on the quality of 



Table 7: AVERAGE HOURLY WAGE BY SEX 



Fema I es 
Ma I es 

Total 



Source 

Sex 

Error 



SS 
183 
2989 



N 


Mean 


St. Dev. 


143 


8.27 




384 


6.78 


2.02 


588 


7.22 


2.35 


df 


MS 


F 


1 1 


183 


35.9 


586 


5.1 





<.001* 



Table 8: AVERAGE HOURLY WAGE BY PROGRAM AREA 





N 


Mean 


Serv i ces 


26 


5.39 


Medical/Heal th 


206 


8.01 


Chi Id Care/Food 


22 


4.65 


Sec ./Bus i ness 


219 


6.58 


Techno 1 ogy 


106 


8.08 


Product i on 


9 


8.46 


Total 


588 


7.22 



S±. Dev. 
1.40~ 
1.64 
1.29 
2.29 
2.64 
3.69 

2.35 



Source 

Program 

Error 



SS 
307 
2798 



df 
5 
583 



MS 
61.4 
4.8 



F 
12.8 



< .001* 



Table 9: CORRELATIONS BETWEEN EFU RATINGS, AVERAGE HOURLY 
WAGE AND SEX <n=588> 



Technical knowledge 
Work Attitude 
Work Qua I i ty 
Techn i ca I Sk ills 
Overal I Rating 
Relative Preparation 



Hourly 
707 
-.01 
-.01 
.05 
-.05 
-.11 



Wage 



<p<.05> 



Sex 
.06 
.04 
.02 
.05 
.01 
.12 



<p<,05> 



wcrk or preparation EFl' survey items (see Table 6>. 

Table 8 presents the average hourly wage of graduates working 
full-time in each program --ea. As can be seen from Table 8, very 
I arge s i gn i f 5 Cc nt d i f f erences were found in average hour I y wage 
between p rogram area. These d i f f e rences i n average hourly wages, 
however, are not correlated to the quality of work or preparation 
ratings of graduates by employers Csee Table 5>. These findings are 
most easily seen by examining the correlations between EFU survey 
items, sex, and average hourly wages. 

Table 9 presents the correlations between the EFU survey items, 
sex, and average hourly wage. As can be seen from Table 9, there are 
no significant correlations between average hourly wage and the EFU 
survey items, or sex and the survev items, with the except? on of 
Relative Preparation which is due to the M Heal/Health programs 
which are almost exclusively female. When the Med i ca I /Hea I th program 
graduates are removed from the sample, the correlations with Relative 
Preparation are non-significant. Average hourly wage, however, is 
significantly a.id strongly related to program area (see Table 8>. 
The quality of a program's graduates as rated by employers, and by 
inference the quality of the program that trained the graduates, 
therefore, are not related to the average hourly wage of program 
graduates, even within a given program area. 

No significant differences were found between females and males 
in EFU ratings by employers, but large significant differences in 
average hourly wages were found, oven within program areas • Th is 
finding is more than somewhat startling as there was a wide age range 
of graduates in this state-wide sample. Age in the sample ranged 
from 22 to 56 with a mean age of 29.2 years. The median age was 28.1 

.1.3 



years. This sample, therefore, contained a high percentage of older 
women and men, and the differences and lack of differences found were 
homogeneous across age levels. It should also be noted that in this 
sample the correlation between ? <je and wage was r=+.34 <p<.001>, 
whereas the correlations between age and EFU ratings were all non- 
significant. Given those facts, it must be said that the findings of 
the present study tend to support the contentions of many that women 
aro not getting equal pay for equal quality or work in the market 
pi ace. 



Qiscussion 

The EFU survey component of the VEDS system was found to be 
highly labor and time intensive and very difficult to implement. 
Employers, however, were very pleased that they were being surveyed 
and were being asked to evaluate graduates. The required VEDS EFU 
instrument used performed reasonably well and was found to be 
reasonably adequate. The average inter-item correlation coefficient 
was r=+.61 and Cronbach alphas ranged from r==.84 to r=+.90 over the 
6 EFU items. Raters, therefore, were consistent but relatively 
discriminating in their judgements. 

Being limited to surveying only those graduates who consented 
to have their employer contacted was not found to be a significant 
empirical problem in the present study, as 87% of the graduates who 
were employed full-time consented to hav i their employer contacted, 
and no significant differences on key VED V /ariables were found 
between those who consented and those who did not. The degree to 
which this finding can be generalized is strictly an empirical 
matter that can only be answered by conducting and analyzing more 
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EFU surveys, particularly as the response rates to the student 
follow-up survey which were high in the present study as the key 
determining factor in terms of both EFU sample size and 
representat iveness. 

In general, Massachusetts public community college graduates 
were rated between good and very good by their employers on the EFU 
survey i terns. S i gn i f i cant d i f f erences in rat i ngs on all cr i ter i a 
were found between graduates working full-time in areas related to 
their training and graduates working full-time in areas not related 
to their training, community colleges, programs, and the same 
programs at different community colleges. These differences not only 
supported the validity of the results obtained, but were also a 
d i reel; stimulus to the conducting of more in-depth studies. 

No significant correlations were found between EFU ratings and 
average hourly wages, even within program areas. Highly significant 
differences in average hourly wages, however, were found between 
major program areas. The quality of a program as measured by 
employers* ratings of its graduates, therefore, was not related to 
the average hourly wages of the program's graduates, oven within 
general program areas. 

Average hourly wage is traditionally considered to be the best 
index of program qual tty and success in the occupati ona I /career 
education area. From the results of the present study, however, 
hourly wage would have to be said to be a very imperfect and highly 
flawed index of program quality and training success as measured 
employer ratings of employee quality. Student follow-up surveys 
alone, therefore, are not sufficient for assessing program quality 
f O and effectiveness, and employer follow-up surveys are also needed 
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for such assessments. 

No significant differences were found between females and males 
in their EFU ratings by employers, but large significant differences 
in average hourly wages were found, even within program areas. This 
sample of graduates <N=588>, which had a median age of 29.2 years, 
confined a high percentage of older men and women and the 
differences and lack of differences were homogeneous across age 
levels. Given these facts, it must be said that these findings tend 
to support current contentions that women are not getting equal pay 
for equal quality or work in the market place. The degree to which 
the findings oV th<? present study would be true of high school or 
four year college graduates or community college graduates in other 
states is an important empirical and theoretical question that needs 
to be answered by further research. 

The employer follow-up component of the VEDS system has been 
strongly opposed by many different people for a wide variety of 
reasons, not the least of which is its labor and time i n tens i veness 
and its difficulty in implementing. The EFU system can only be 
implemented by implementing a graduate unit-record system. Even 
given these difficult factors, the employer follow-up component of 
the VEDS system would seem to bo a worthwhi la and needed endeavor 
relative to assessing graduate and program quality adequately, as 
well as to assessing progress on equity issues in the market place 
as opposed to the schools. 
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